Analysis of fMRI data sampled from Large Populations: Statistical and Methodological Issues

Validating the association between brain activity, as measured in functional MRI, with a combination or a contrast of tasks is usually performed by replicating an experiment in a small group of subjects, and by assessing the presence of a statistically significant average effect across subjects (random effects analyses). While many efforts have been made to control the rate of false detections, statistical characteristics of the data have rarely been studied, and the reliability of the results (supra-thresholds areas that are considered as activated regions) has rarely been assessed. In this work, we take advantage of the large cohort of subjects who underwent the Localizer experiment to study the statistical nature of group data, propose some measures of the reliability of group studies, and address simple methodological questions as : is there, from the point of view of reliability, an optimal statistical threshold for activity maps ? How many subjects should be included in group studies ? What method should be preferred for inference ? Our results suggest that i) optimal thresholds can indeed be found, and are rather lower than usual corrected for multiple comparison thresholds ii) 20 subjects or more should be included in functional neuroimaging studies in order to have sufficient reliability, iii) non-parametric significance assessment should be preferred to parametric methods iv) cluster-level thresholding is more reliable than voxel-based thresholding v) mixed effects tests are much more reliable than random effects tests. Moreover, our study shows that inter-subject variability plays a prominent role in the relatively low sensitivity and reliability of group studies.